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WHAT IS CLAIMED IS: 

1 . A method for detecting a semantic temporal event, said method comprising: 

retrieving multiple-layer models corresponding to said semantic temporal 

event; 

receiving temporal observations that are extracted, from at least one data 
source, according to said multiple-layer models for the semantic temporal event; and 

detecting one or more occurrences of the semantic temporal event based on 
said temporal observations and said multiple-layer models. 

2. The method according to claim 1, wherein said semantic temporal event includes a 
sports event. 

3. The method according to claim 2, wherein said sports includes a soccer game. 

4. The method according to claim 1, wherein said multiple-layer models include a 
high level domain-specific knowledge model and a dynanric hierarchical event model. 

5. The method according to claim 4, wherein said high level domain-specific 
knowledge includes rules of a sports game. 

6. The method according to claim 4, wherein said dynamic hierarchical event model 
includes a hierarchical decision tree. 

7. The method according to claim 4, wherein said dynamic hierarchical event model 
includes an entity-relationship-diagram. 

8. The method according to claim 1, wherein said data source includes data 
acquisition devices such as a camera, a microwave sensor, and a sound recorder, and input 
data stream such as video, audio, text, and temporal features. 
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9. The method according to claim 8, wherein said temporal features include 
tracking position data. 

1 0. The method according to claim 1 , wherein said detecting comprises: 
performing semantic temporal event detection using at least one detection 

method, each of said at least one detection method producing a detection result; and 

combining, if more than one of said at least one detection method is used in 
said performing, said detection result from said at least one detection method to produce a 
final detection. 

11. The method according to claim 10, wherein said at least one method includes 
dynamic Bayesian networks, rule based expert system, decision trees. Hidden Markov 
Models, neural networks, or fuzzy logic. 

12. The method according to claim 1, further comprising: 

characterizing said occurrences of the semantic temporal event, detected by 
said detecting, to produce a characterization; and 
storing said characterization. 

13. The method according to claim 1 2, wherein said characterization includes: 
a set of statistics computed from said occurrences; or 

a set of descriptions, each of which describes an action happening in an 
occurrence of the semantic temporal event. 

1 4. The method according to claim 12, further comprising: 
performing temporal event prediction based on said characterization; 
revising said multiple-layer models for said semantic temporal event based on 

said characterization; and 
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simulating parts of said semantic temporal event according to said 
characterization. 

15. A system for detecting a semantic temporal event, said system comprising: 

a knowledge-based modeling unit for generating multiple-layer models for said 
semantic temporal event; 

a storage mechanism for storing said multiple-layer models; 

an observation collection unit for extracting, from at least one data source, 
temporal observations according to said multiple-layer models for the semantic temporal 
event; and 

a temporal event detection unit for detecting one or more occurences of the 
semantic temporal event based on said temporal observations and said multiple-layer models. 

16. The system according to claim 15, further comprising: 

an event characterization unit for characterizing said occurrences of the 
semantic temporal event, detected by said temporal event detector, to produce a 
characterization for the occurrences of the semantic temporal event. 

17. The system according to claim 16, further comprising: 

a storage mechanism for storing the characterization produced by said event 
characterization unit; 

an event prediction unit for performing temporal event prediction based on said 
characterization; 

an event model updating unit for modifying said multiple-layer models based 
on said characterization; and 
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an event simulation unit for simulating parts of said semantic temporal event 
according to said characterization. 

18. A computer-readable medium encoded with a program for detecting a semantic 
temporal event, said program comprising: 

retrieving multiple-layer models corresponding to said semantic temporal 

event; 

receiving temporal observations that are extracted, from at least one data 
source, according to said multiple-layer models for the semantic temporal event; and 

detecting one or more occurrences of the semantic temporal event based on 
said temporal observations and said multiple-layer models. 

19. The medium according to claim 18, wherein said semantic temporal event 
includes a sports temporal event. 

20. The medium according to claim 18, wherein said sports includes a soccer 

game. 

21. The medium according to claim 18, wherein said multiple-layer models 
include a high level domain-specific knowledge model and a dynamic hierarchical event 
model. 

22. The medium according to claim 21, wherein said high level domain-specific 
knowledge includes rules of a sports game. 

23. The medium according to claim 1 , wherein said detecting comprises: 
performing semantic temporal event detection using at least one detection 

method, each of said at least one detection method producing a detection result; and 
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combining, if more than one of said at least one detection method is used in 
said performing, said detection resuh from said at least one method to produce a detection. 

24. The medium according to claim 23, wherein said at least one method includes 
dynamic Bayesian networks, rule based expert system, decision trees. Hidden Markov 
Models, neural networks, or fuzzy logic. 

25. The medium according to claim 1 8, said program further comprising: 
characterizing said occurrences of the semantic temporal event, detected by 

said detecting, to produce a characterization; and 
storing said characterization. 

26. The medium according to claim 25, wherein said characterization includes: 
a set of statistics computed from said occurrences; or 

a set of descriptions, each of which describes an action happening in an 
occurrence of the semantic temporal event. 

27. The medium according to claim 25, said program further comprising: 
performing temporal event prediction based on said characterization; 
revising said hierarchical event model based on said characterization; and 
simulating parts of said semantic event according to said characterization. 

28. A method of querying semantic temporal events, said method comprising: 
receiving a query request from a cHent at a search engine server; 

retrieving data requested by said client from a data storage based on event 
based indexing; and 

sending said data retrieved by said trtrieving to said client. 

29. The method according to claim 28, wherein 
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said query request includes a semantic temporal event; 

said query request includes statistics of the occurrences of a semantic temporal 

event; 

said query request includes statistics of a sports game; and 

said query request includes descriptions of actions in a sports game. 
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